Corrections for recognizers

ABSTRACT

A processing device may recognize a number of input handwritten strokes, which may represent a mathematical expression, a chemical formula, or other two-dimensional structure. Rewriting rules of a grammar may be applied to the strokes to produce a number of possible recognition results. Each of the possible recognition results has a respective score based on a sum of rewriting rules applied to the strokes to produce respective ones of the possible recognition results. Input may be provided to identify misrecognized strokes and a correct terminal production, or symbol corresponding to the misrecognized strokes. Strokes may be misrecognized for many reasons, including parsing errors, over-grouping or under-grouping of matrices, and improper placement of a recognized terminal production, or symbol, with respect to a root structure. Correction hints may be leveraged for correcting types of errors mentioned above.

BACKGROUND

Many state-of-the-art processing devices, such as tablet personalcomputers (PCs) or other processing devices, permit a user to provideinput as digital ink. The digital ink may include one or morehandwritten strokes, which a processing device may display as ink whilethe one or more handwritten strokes are being input. A handwrittenstroke may begin when a writing instrument lands on a writing surface,and may end when the writing instrument is lifted off the writingsurface. The writing surface may include a display screen, which mayfurther include a digitizer, and the writing instrument may be anelectronic or non-electronic pen, a stylus, a user's own finger, apointing device, such as, for example, a computer mouse, or anotherwriting instrument.

A user may input one or more handwritten strokes, as digital ink, toform a two-dimensional structure. The two-dimensional structure mayinclude a mathematical expression, a chemical formula, textualcharacters, numbers, or other two-dimensional structures.

Existing recognizers may produce a recognition result having amisrecognized portion. The user may attempt to correct the misrecognizedportion by selecting the misrecognized portion of the recognitionresult. In at least one existing recognizer, the user may attempt tocorrect a misrecognized character of the recognition result by selectingthe digital ink corresponding to the misrecognized character. Selectingthe misrecognized portion of the recognition result, or selecting thedigital ink corresponding to the misrecognized character in therecognition result, may cause one or more alternates to be displayed.The user may select one of the one or more alternates, thereby causingthe misrecognized character to be replaced with the selected one of theone or more alternates without re-recognizing the input digital ink. Oneexisting recognizer permits the user to type a correct character from akeyboard when the correct character is not among the displayed one ormore alternates.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that is further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

In embodiments consistent with the subject matter of this disclosure, amethod and a processing device are provided for correcting a recognizedhandwritten two-dimensional structure. A number of handwritten strokes,or atoms may be received and recognized to produce a displayedrecognition result. Rewriting rules of a grammar may be applied to termsof the grammar to produce non-terminal productions and terminalproductions. A score may be assigned to each of the rewriting rules by arecognizer. A score of each possible recognition result may be arespective sum of scores of rewriting rules applied to produce eachrespective one of the possible recognition results. A possiblerecognition result having a best score may be selected as a recognitionresult to display.

Recognition results may have errors for a number of reasons. Embodimentsconsistent with the subject matter of this disclosure provide a facilityfor indicating errors and corrections for the errors (hereinafter,referred to as correction hints). The correction hints may be leveragedduring re-recognition of the atoms, such that a correct recognitionresult may be produced and displayed. Correction hints may be providedfor correcting misrecognized symbols, digital ink parsing errors, errorsthat occur when recognizing particular structures such as, for example,matrices, and mathematical expressions including root structures, andany combination of the above.

DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionis described below and will be rendered by reference to specificembodiments thereof which are illustrated in the appended drawings.Understanding that these drawings depict only typical embodiments andare not therefore to be considered to be limiting of its scope,implementations will be described and explained with additionalspecificity and detail through the use of the accompanying drawings.

FIG. 1 illustrates a functional block diagram of an exemplary processingdevice, which may implement embodiments consistent with subject matterof this disclosure.

FIGS. 2-18 illustrate a number of correction examples, which may beperformed in embodiments consistent with the subject matter of thisdisclosure.

FIGS. 19-24 are flowcharts illustrating exemplary processing which maybe performed in embodiments consistent with the subject matter of thisdisclosure.

DETAILED DESCRIPTION

Embodiments are discussed in detail below. While specificimplementations are discussed, it is to be understood that this is donefor illustration purposes only. A person skilled in the relevant artwill recognize that other components and configurations may be usedwithout parting from the spirit and scope of the subject matter of thisdisclosure.

Overview

Embodiments consistent with the subject matter of this disclosure mayprovide a method and a processing device for correcting a recognizedhandwritten two-dimensional structure, which may include a mathematicalexpression, a chemical formula, or another two-dimensional structure.

A user may input digital ink as a number of strokes, or atoms. Theprocessing device may process the input digital ink to produce arecognition result, which may be presented, or displayed, to a user. Theprocessing device may have one or more recognizers for scoringnon-terminal productions, or structures, and terminal productions, orsymbols. In this application, the term “terminal production refers to agrammar terminal production, or symbol, which is a non-divisible grammarproduction. Examples of terminal productions may include, but not belimited to, letters that denote variable names, digits, mathematicaloperators, mathematical functions (such as sin, cos, arcsin, etc) andsymbols.

A grammar may include a number of rewriting rules for producingnon-terminal and terminal objects, or productions. The rewriting rulesmay be applied to terms of the grammar formed by one or more atoms in aregion to produce non-terminal and terminal productions. As a rewritingrule is applied to a region, the region may be partitioned to producemultiple subregions. The rewriting rules may be applied until all atomsare recognized as being included in terminal productions. As a rewritingrule is applied to the one or more atoms, a score may be associated withthe applied rewriting rule. Thus, for example, the score for aparticular recognition result may be a sum of rewriting rules applied toproduce the recognition result. The rewriting rules may be applied tothe one or more atoms in a number of different ways to produce multiplepossible recognition results. One of the possible recognition resultshaving a best score may be selected as the recognition result. Invarious embodiments, a best score may be a highest score among scores ofpossible recognition results. In other embodiments, a best score may bea lowest score among the scores of possible recognition results.

Occasionally, recognition results may have a misrecognized portion.Misrecognition of atoms may occur for a number of reasons, such as, forexample poor handwriting and ink parsing errors. For example, atoms maybe input in such a way that atoms of different terminal productions mayoverlap, or cross one another, such that the atoms may not be properlyrecognized. Embodiments consistent with the subject matter of thisdisclosure, may provide a method and a processing device for providing acorrection hint for correcting ink parsing of the atoms. The atoms maythen be re-recognized taking the correction hint into consideration.

Further, atoms included in some structures may cause recognition errorsdue to placement of the atoms within the structures. For example, atomsrepresenting an expression including multiple symbols may be intended tobe included under a root symbol (√{square root over ( )}). However, theatoms may be entered such that a first portion of the atoms may beincluded under the root symbol, while a second portion of the atoms maynot be included under the root symbol. As a result, some symbols of theexpression may be recognized as being under the root symbol, whileothers symbols of the expression may be recognized as being outside ofthe root symbol. Further, atoms representing symbols within a matrix maybe placed such that the matrix may be misrecognized as having too manyrows or columns, or too few rows or columns. Embodiments consistent withthe subject matter of this disclosure may provide a method and aprocessing device for providing a facility for a user to correct errorswith respect to a root symbol or a matrix.

Exemplary Processing Device

FIG. 1 is a functional block diagram of an exemplary processing device100, which may be used in embodiments consistent with the subject matterof this disclosure. Processing device 100 may include a bus 110, aninput device 160, a memory 130, a read only memory (ROM) 140, an outputdevice 150, a processor 120, and a storage device 150. Bus 110 maypermit communication among components of processing device 100.

Processor 120 may include at least one conventional processor ormicroprocessor that interprets and executes instructions. Memory 130 maybe a random access memory (RAM) or another type of dynamic storagedevice that stores information and instructions for execution byprocessor 120. Memory 130 may also store temporary variables or otherintermediate information used during execution of instructions byprocessor 120. ROM 140 may include a conventional ROM device or anothertype of static storage device that stores static information andinstructions for processor 120. Storage device 150 may include compactdisc (CD), digital video disc (DVD), a magnetic medium, or other type ofstorage medium for storing data and/or instructions for processor 120.

Input device 160 may include a keyboard, a touchscreen, or other inputdevice. If input device 160 includes a touchscreen, the touchscreen mayfurther include a digitizer for receiving input from a writing device,such as, for example, an electronic or non-electronic pen, a stylus, auser's finger, or other writing device. In one embodiment, input device160 may include a pointing device, such as, for example, a computermouse, or other pointing device. Output device 170 may include one ormore conventional mechanisms that output information, including one ormore display monitors, or other output devices.

Processing device 100 may perform such functions in response toprocessor 120 executing sequences of instructions contained in atangible machine-readable medium, such as, for example, memory 130, ROM140, storage device 150 or other medium. Such instructions may be readinto memory 130 from another machine-readable medium or from a separatedevice via a communication interface (not shown).

Grammar Parsing Framework

A grammar parsing framework is a method used when recognizinghandwritten or printed mathematical equations, chemical formulas, textstructures, and graphs, as well as other two-dimensional structures.

A document may be defined as an object subject to recognition, such as,for example, a handwritten mathematical expression, handwritten text, ascanned image of a printed mathematical expression or printed text, ahandwritten page of ink drawings, text and diagrams, as well as otherhandwritten or scanned two-dimensional structures. FIG. 2 shows anexemplary document including a handwritten mathematical expression.

Recognition may be defined as a process of transforming a document intoa form understandable by a machine. For example, a handwrittenmathematical expression may be transformed to plain text in a MathMLformat, a Latex format, or another format. The transformed document maybe read and searched by a machine and may be suitable for storing andindexing.

A document atom may be defined as an indivisible part of a document anda document may be a collection of atoms. If a document is written indigital ink, then an atom of the document may be a single stroke of thedigital ink. In FIG. 3, each atom is circled. As one can see, themathematical expression of FIGS. 2 and 3 includes 10 atoms.

A document region may be defined as a collection of atoms from a givendocument. A region R containing atoms a₁, a₂, and a₄ may be written asR={a₁,a₂,a₄}. A size of a region R may be defined as a number of atomsin the region and may be written as |R|. A maximal document region maybe defined as a region that contains all atoms from a document. Amaximal document region R may be written as R=D.

A document partition may be defined as a pair of document regions,written as P=(R₁,R₂), such that R₁ and R₂ have no atoms in common(R₁∩R₂=0) Partition P belongs to a region R if (R₁∪R₂=R).

A binary context-free grammar (hereinafter referred to as grammar) maybe defined as a triplet G=(T, S, P) where:

-   -   1. T is a finite set of terms, T={t₁,t₂, t₃, . . . , t_(n)};    -   2. S is a starting term, such that SεT;    -   3. P is a finite set of productions, P={p₁,p₂,p₃, . . . ,        p_(k)}.        -   a. A production may be one of: a binary production, a unary            production or a terminal production.        -   b. A binary production may be defined as a triplet of terms,            p=(t_(i),t_(j),t_(k)), written as:            -   t_(i)→t_(j)t_(k)        -   c. A unary production may be defined as a pair of terms,            p=(t_(i),t_(j)), written as:            -   t_(i)→t_(j)        -   d. A terminal production may be defined as a singleton of a            term, p=(t_(i)), written as:            -   t_(i)→

All three forms of productions may be written simply as(t_(i),t_(j),t_(k)) by allowing t_(j) and t_(k) to be empty sets.Therefore, a unary production may be written as t_(i)→t_(j)0 and aterminal production may be written as t_(i)→00.

Given a grammar G=(T, S, P), words may be defined as all n-tuples withelements from the set of terms T.

Word Examples May Include:

-   w₁=(t₂,t₁)-   w₂=(t₂)-   w₃=(t₅,t₅,t₁,t₅)

A rewriting step may be defined as a transformation of one word intoanother word by applying a rewriting rule. Word w₁ being transformedinto word w₂, may be written as w₁→w₂, using production A→B C as arewriting rule if: w₁=(α, A,β)→w₂=(α,B,C,β)

-   w₁=(α,A,β)    w₂=(α,B,C,β)

For example, rewriting steps, which follow, with annotated rewritingrules may be performed:

(a, x, d) → (a, b, c, d) x → b c (a, x, d) → (a, y, d) x → y (a, x, d) →(a, d) x →

A sequence of rewriting steps may be defined as a procedure ofperforming rewriting steps one by one. The sequence of rewriting stepsmay be written as:

-   w₁→w₂→w₃→w₄→□

Terminal words in a grammar G may be defined as all words w₁=(t₁, t₂, .. . , t_(k)) such that t_(i)→ are terminal productions from G.

Grammar words may be defined as all terminal words in a given grammar Gthat exist in some sequence of rewriting steps starting from a wordw_(s)=(S) (a starting word may consist only of a grammar starting term).If word w is a grammar word, this may be written as:

-   -   (s)→*w_(s)

A process of pruning regions and partitions from a document may becalled document parsing. If the document includes handwritten ink, theprocess may be called ink parsing.

RECOGNITION EXAMPLE

A simple exemplary grammar for recognition of numbers and basicarithmetic operations (called numbers grammar) may be defined asfollowing:

-   -   1) G=(T, S, P)    -   2) T={Math, Number, Op, OpNumber, DigitList, Digit, Digit9, 0,        1, 2, 3, 4, 5, 6, 7, 8, 9, +, −}    -   3) S=Math    -   4) P={        -   Math→Number,        -   Math→Math OpNumber,        -   OpNumber→Op Number,        -   Op→+,        -   Op→−,        -   +→,        -   −→,        -   Number→Digit,        -   Number→Digit9 DigitList,        -   DigitList→Digit,        -   DigitList→DigitList→Digit,        -   Digit→Digit9,        -   Digit→0,        -   Digit9→1,        -   Digit9→2,        -   Digit9→3,        -   Digit9→4,        -   Digit9→5,        -   Digit9→6,        -   Digit9→7,        -   Digit9→8,        -   Digit9→9,        -   0→,        -   2→,        -   3→,        -   4→,        -   5→,        -   6→,        -   8→,        -   9→,        -   }

Production names in the simple exemplary grammar are almost allself-explanatory; production Digit9 may represent all digits but zero,production Digit9 exists to ensure that words like ‘012’ may not begenerated by the grammar.

The exemplary grammar is a simple grammar which is able to generatewords like: 0, 12, 0+12−2000, 1+2+3, etc. A full-size grammar thatsupports university level mathematics may have thousands of productionsand terms.

An exemplary document to recognize is illustrated in FIG. 4. In FIG. 4,every atom is assigned an identifier (1, 2, 3, 4, 5). A sequence ofrewriting steps, or rules that may be used to recognize the exemplarydocument is as follows:

1. (Math, {1, 2, 3, 4, 5})

-   -   a. apply rule: Math→Math OpNumber, {1, 2, 3, 4, 5}→{1} {2, 3, 4,        5}

2. (Math, {1}), (OpNumber, {2, 3, 4, 5})

-   -   a. apply rule: Math→Number, {1}→{1}

3. (Number, {1}), (OpNumber, {2, 3, 4, 5})

-   -   a. apply rule: Number→Digit, {1}→{1}

4. (Digit, {1}), (OpNumber, {2, 3, 4, 5})

-   -   a. apply rule: Digit→0, {1}→{1}

5. (0, {1}), (OpNumber, {2, 3, 4, 5})

-   -   a. 0→is a terminal production    -   b. apply rule: OpNumber→Op Number, {2, 3, 4, 5}→{2, 3} {4, 5}

6. (0, {1}), (Op, {2, 3}), (Number, {4, 5})

-   -   a. apply rule: Op→+, {2, 3}→{2, 3}

7. (0, {1}), (+, {2, 3}), (Number, {4, 5})

-   -   a. +→is a terminal production    -   b. apply rule: Number→Digit9 DigitList, {4, 5}→{4} {5}

8. (0, {1}), (+, {2, 3}), (Digit9, {4}), (DigitList, {5})

-   -   a. apply rule: Digit9→2, {4}→{4}

9. (0, {1}), (+, {2, 3}), (2, {4}), (DigitList, {5})

-   -   a. 2 is a terminal production    -   b. apply rule: DigitList→Digit, {5}→{5}

10. (0, {1}), (+, {2, 3}), (2, {4}), (Digit, {5})

-   -   a. apply rule: Digit→3, {5}→{5}

11. (0, {1}), (+, {2, 3}), (2, {4}), (3, {5})

-   -   a. 3→is a terminal production

12. Recognition finished.

A recognition result from the above example is ‘0+23’. Symbol ‘0’ isassigned with region {1} (i.e. atom with identifier 1 is recognized assymbol ‘0’). Symbol ‘+’ is assigned with region {2, 3} (i.e. a group ofatoms with identifiers 2 and 3 is recognized as symbol ‘+’). Symbol ‘2’is assigned with region {4} and symbol ‘3’ is assigned with region {5}.

There are many combinations of rewriting rules that may be used torecognize the exemplary document of FIG. 4. A recognition engine mayassign scores for each of the rewriting rules. A score for a particularrecognition result may be a sum of all scores for all rewriting rulesused to produce the particular recognition result. A best recognitionresult may be a recognition result having a best score. Typically, thebest score is a highest score. Although, in some embodiments, a bestscore may be a lowest score.

A following formula may be used to find a best recognition result:

$\begin{matrix}{{C\left( {A,R_{0}} \right)} = {{\max\limits_{\underset{\underset{{R_{1}\bigcup R_{2}} = R_{0}}{{R_{1}\bigcap R_{2}} = 0}}{A->{BC}}}{C\left( {B,R_{1}} \right)}} + {C\left( {C,R_{2}} \right)} + {{score}\left( {{A->{BC}},R_{0},R_{1},R_{2}} \right)}}} & {{Equ}.\mspace{14mu} 1}\end{matrix}$where score( ) may be a score assigned to a rewriting rule and C(A, R)may be a best result, so far, for a given term, A, on a given region, R.

Score( ) may be defined for binary, unary, and terminal productions. Forbinary productions, score may have a form as shown above in Equ. 1. Forunary productions, score may have a form

score(A→B 0, R₀,R₁,0).

For terminal productions, score may have a form

score(A→00, R₀,0,0).

Correction Feature

FIG. 5 illustrates a handwritten mathematical expression 502 and acorresponding recognition result 504. Recognition result 504 has amisrecognized portion 506. Recognition result 504 may be obtained byapplying multiple combinations of a number of rewriting rules to inputatoms, summing scores of applied rewriting rules for each of themultiple combinations, and producing possible recognition results andrespective scores for each of the multiple combinations. One of thecombinations having a best score, according to Equ. 1, may be selectedas recognition result 504.

A user may select one or more atoms, corresponding to misrecognizedportion 506, by using a selection tool, such as, for example, a lassoselection tool, to draw a lasso around the one or more atoms. Theselection tool may be used via a pointing device, such as, for example,a computer mouse, an electronic pen, a stylus or a user's finger on atouchscreen having a digitizer, or via other input methods.Alternatively, instead of using a lasso selection tool, anotherselection tool may be used. FIG. 5 shows selected atoms surrounded by arectangle 508, which may define a region.

Upon selecting the one or more atoms, a recognizer may recognize the oneor more atoms in isolation, in the region defined by the selection tooland may add results of recognizing the one or more atoms in isolation toa previously determined list of alternate recognition results for theregion. The recognizer may then provide, or display, several recognitionresults associated with top best recognition scores. The severalrecognition results may be called possible or alternate recognitionresults. The user may select one of the alternate recognition resultsusing a selection tool. In FIG. 5, the user selected, from a first menu510, “tan”, which may be highlighted. In this example, selection of“tan” from the first menu may cause a second menu 512 to be presented.Second menu may have a number of alternates related to “tan”. In thisexample, the user selected “tan” from second menu 512 using theselection tool. The selection of “tan” from second menu 512 may becalled a correction hint. A correction hint is an element from analternate list generated over a region R that is selected by a user andprovided to a recognizer. Thus, a correction hint is a rewriting rule(A,R)→(B,R₀)(C,R₁), where (A,B,C) is a grammar production, which may bewritten as A→BC, and (R₀, R₁) is a document partition (R=R₀∪R₁,R₀∩R₁=0).

The correction feature may be implemented by replacing Equ. 1 with

$\begin{matrix}{{C\left( {A,R_{0}} \right)} = {{\max\limits_{\underset{\underset{{R_{1}\bigcup R_{2}} = R_{0}}{{R_{1}\bigcap R_{2}} = 0}}{A->{BC}}}{C\left( {B,R_{1}} \right)}} + {C\left( {C,R_{2}} \right)} + {{score}\; 2\left( {{A->{BC}},R_{0},R_{1},R_{2}} \right)}}} & {{Equ}.\mspace{14mu} 2}\end{matrix}$where score2 is a new scoring function. Score2 may determine whether arewriting rule respects all correction hints. If the rewriting rulerespects all correction hints, then a score for the rewriting rule maybe equal to a score using the score function. Otherwise, a penalty scoremay be added to an assigned score for the rewriting rules. For example,if a best recognition result has a highest score, then if a rewritingrule does not respect all correction hints, score2 may add the penaltyscore to the score assigned to the rewriting rule. The penalty score maybe a large negative value, such as, for example, −100, or anothersuitable value. In an embodiment in which a best recognition result hasa lowest score, the penalty score may be a large positive value, suchas, for example, 100, or another suitable value. Adding the penaltyscore to the score of a possible recognition result may insure is thatthe possible recognition result will not have a best score and will notbe selected as a best recognition result.

Returning to FIG. 5, after providing the correction hint, input atomsmay be re-recognized and a correct recognition result 602 may beproduced, as shown in FIG. 6.

Ink Parsing Errors

A number of possible regions and partitions in a document may beexponential. Ink parsing is a process by which regions and partitionsmay be pruned to a number more suitable for computation. Ink parsing mayprune as many regions as possible, while not pruning regions andpartitions that are part of a correct, or expected, recognition result.

If, for some reason a region that is part of a correct recognitionresult is pruned from a document, the document cannot be correctlyrecognized. The previously-discussed correction feature cannot correctink parsing errors. After receiving a correction hint and re-recognizinginput atoms, a region that is part of a correct recognition result mayagain be pruned during ink parsing.

Ink parsing errors may be corrected when using a simple ink parsingalgorithm having a rule, such that, on a region R, partition P=(R₀,R₁)is valid (R=R₀∪R₁,R₀∩R₁=0) if and only if regions R₀ and R₁ areseparable by a horizontal or a vertical line. Ink parsing errors mayalso be corrected when using a more complicated ink parsing algorithm.

Ink parsing errors may be corrected by shrinking regions based oncorrection hints. A correction hint may be written as a rewriting rule(A,R₀)→(B,R₁)(C,R₂). A main region R₀ may be shrunk such that aprobability of the region being pruned may be decreased. As a result ofshrinking the region R₀, boundaries of the region R₀ may be changed andatoms inside the region R₀ may be linearly transformed to fit new regionboundaries.

FIG. 7 illustrates an ink parsing error and correction of the inkparsing error. A user may input atoms 702, which may be misrecognized asrecognition result 704 due to an ink parsing error (note that a regionbounding a first atom of input atoms 702 may not be separable from aregion bounding a second atom of input atoms 702 by a horizontal or avertical line). The user may select a region of input atoms 702 using aselection tool. A bounding box 706 showing the region may be displayed.A bounding box may be defined as a minimal rectangle whose sides areparallel to an x and a y axis of a plane which contains a whole regionor a whole document atom.

A recognizer may attempt to recognize one or more atoms within boundingbox 706. In the example of FIG. 7, the recognizer may attempt torecognize atom 708, as may be indicated by a visual indication such as,for example, displaying atom 708 with a bold line. The recognizer mayrecognize atom 708, within a region defined by bounding box 706, inisolation, and may present a best recognition result and severalalternate recognition results included in a menu 710. The user mayselect one of the recognition results 712, which may then behighlighted. A shrunken bounding box 714 and transformed input atom 716are shown such that a bounding box of transformed input atom 716 may beseparable from a bounding box of any other input atoms which wereoriginally in bounding box 708. Input atoms 702 may now be correctlyrecognized to produce correct recognition result 718.

FIG. 8 illustrates another ink parsing correction example. Therecognizer may incorrectly parse input atoms 802 such that recognitionresult 804 is incorrect. Using a selection tool, a user may select aregion 806 corresponding to the ink parsing error. Region 806 may beshown by a bounding box surrounding one or more atoms. In this example,the bounding box surrounds atom 808, which is shown in bold. Therecognizer may recognize atom 808 in isolation and may provide, ordisplay, a menu 810, including a best recognition result and severalalternate recognition results with respect to atom 808. In this example,the user may select recognition result 812 as a correction hint. Region806 may then be shrunk in order to separate atom 808 from any otheratoms that were originally in region 806. Atom 808 may be transformed inproportion to shrunken region 806. Atoms 814 may then be re-recognizedto produce correct recognition result 816.

FIG. 9 illustrates a third ink parsing correction example. Therecognizer may incorrectly parse input atoms 902 such that recognitionresult 904 is incorrect. Using a selection tool, a user may select aregion 906 corresponding to the ink parsing error. Region 906 may beshown by a bounding box surrounding one or more atoms. In this example,the bounding box surrounds atom 908, which is shown in bold. Therecognizer may recognize atom 908 in isolation and may provide, ordisplay, a menu 910, including a best recognition result and severalalternate recognition results with respect to atom 908. In this example,the user may select recognition result 912 as a correction hint. Region906 may then be shrunk in order to separate atom 908 from any otheratoms that were originally in region 906. Atom 908 may be transformed inproportion to shrunken region 906. Atoms 914 may then be re-recognizedto produce correct recognition result 916.

FIG. 10 illustrates a fourth ink parsing correction example. Therecognizer may incorrectly parse input atoms 1002 such that recognitionresult 1004 is incorrect. Using a selection tool, a user may select aregion 1006 corresponding to the ink parsing error. Region 1006 may beshown by a bounding box surrounding one or more atoms. In this example,the bounding box surrounds atoms 1008, shown in bold. The recognizer mayrecognize atoms 1008 in isolation and may provide, or display, a firstmenu 1010 including a form of a best recognition result and severalalternate recognition results with respect to atoms 1008. In thisexample, the user may select form 1012. In response to a selection ofform 1012, several possible recognition results corresponding to form1012 may be presented. The user may then select recognition result 1013as a correction hint. Region 1006 may then be shrunk in order toseparate atoms 1008 from any other atoms that were originally in region1006. Atoms 1008 may be transformed in proportion to shrunken region1006. Atoms 1014 may then be re-recognized to produce correctrecognition result 1016.

Matrix Correction Features

Mathematical structures may have specific correction features based onleveraging information from correction hints. The correction hints maybe utilized as much as possible, such that errors, which may not becorrectable via other methods, maybe corrected with specific correctionfeatures for some structures.

Matrix correction features may correct a number of rows and/or a numberof columns of a matrix. In addition to using matrix correction featuresfor matrices, other correction features, previously discussed, may alsobe used for matrices, as well as other mathematical structures.

As previously discussed, every correction hint may be given as arewriting rule. Rules for correcting matrices may include:

-   -   1. Correction of an over-grouping error (too many columns or        rows recognized): If R₀ is a main region in a correction hint,        then if any atom of the region R₀ belongs to matrix cell, then        all atoms of the region R₀ are to be inside that same cell.        Otherwise, the matrix may be invalid and cannot be recognized.    -   2. Correction of an under-grouping error (too few columns or        rows recognized): If R₀ is the main region in a correction hint,        then a matrix in which R₀ is a complete cell (that is, the cell        that contains the whole region R₀ and no other atoms) may be        created. The corrected matrix may not be a final recognition        result. However, a possibility that the corrected matrix is the        final recognition result may be checked and evaluated.

Rule 1 of the matrix correction feature is deterministic and mayautomatically be respected. Rule 2 represents one possible recognitionresult which may not be a best result. An example of when rule 2 may notbe respected is when a correction is made inside a matrix cell. In thisexample, the correction doesn't mean matrix cells should be divided intomultiple matrix cells.

FIG. 11 shows an example of the matrix correction feature. Input atoms1102 may be recognized to produce a recognition result 1104, which is amatrix having 2 rows and 2 columns. A user may select atoms 1106 using aselection tool to define a region including atoms 1106. As a result,atoms 1106 may be recognized in isolation to produce possiblerecognition results, the possible recognition results may be added topreviously determined possible recognition results for the region, and afirst menu 1108 may be displayed and may include one or more of thepossible recognition results. The user may select result 1110, which maycause a second menu 1112, which may include several possible alternaterecognition results corresponding to result 1110, to be displayed. Theuser may then select alternate recognition result 1114 indicating that“2y” is to be recognized as part of a single matrix cell. Thus, rule 1of the matrix correction feature is satisfied in that the correctionhint indicates that atoms 1106 are to be recognized as part of a singlematrix cell. Input atoms 1102 may be re-recognized and the correctionhint may be considered such that a correct recognition result 1202 (FIG.12) may be produced to correct an over-grouping error.

FIG. 13 shows a second example of the matrix correction feature. Inputatoms 1302 may be recognized as a recognition result 1304, which is avalue within parentheses (a 1 by 1 matrix). A user may select atom 1306using a selection tool. As a result, atom 1306 may be recognized inisolation to produce possible recognition results, the possiblerecognition results may be added to previously determined possiblerecognition results for a same region, and a menu 1308 may be displayedincluding several possible alternate recognition results correspondingto atom 1306. The user may then select alternate recognition result 1314indicating that “δ” may represent contents of a complete matrix cell. Inother words, rule 2 of the matrix correction feature is satisfied. Acorrection hint indicating that atom 1306 corresponds to “δ” may beconsidered during a re-recognition of input atoms 1302. Input atoms 1302may be re-recognized and the correction hint may be considered andevaluated when selecting a recognition result from among possiblerecognition results. FIG. 14 illustrates a correct recognition result1402 (FIG. 14) being produced to correct the under-grouping error shownin FIG. 13.

As mentioned previously, other error correction techniques may be usedwith mathematical structures. FIG. 15 illustrates an example in whichanother error correction technique may be used to correct a matrixstructure.

Input atoms 1502 may include atoms 1504, which may be incorrectlyparsed. With a selection tool, a user may select a region 1506 includingatom 1504. Atom 1504, indicated in bold, may be recognized in isolationto produce possible recognition results, the possible recognitionresults may be added to previously determined possible recognitionresults for the region, and a menu 1508 may be presented to the user.The user may select a possible recognition result 1510, corresponding toatom 1504, resulting in region 1506 being shrunk and atom 1504 beingtransformed in proportion to shrunken region 1506, such that a regionincluding atom 1504 may be separable from a region including anotheratom of original region 1506, as shown in FIG. 16. Atoms 1612 may bere-recognized to produce a correct result 1602, shown in FIG. 16.

Root Structure Correction Features

Root structure correction features are related to correction of nth rootstructures and square root structures. Other correction featuresdiscussed above, except for matrix correction features, may be appliedto square root structures, nth root structures, as well as othermathematical structures.

Root structure correction features may be based on correction hints. Asmentioned previously, every correction hint may be given as a rewritingrule. The rules for the root structure correction feature may include:

-   -   1. If any stroke from a region R₀ is inside an expression under        a root symbol, then all strokes from the region R₀ are to be        included in the expression under the root symbol. This rule is        for both n-th root structures and square root structures.

FIG. 17 illustrates the root structure correction features. Input atoms1702 may be misrecognized and incorrect recognition result 1706 may beproduced and displayed. Using a selection tool, a user may select aregion including atoms 1704 corresponding to a misrecognized portion ofincorrect recognition result 1706 to produce possible recognitionresults, the possible recognition results may be added to previouslydetermined possible recognition results for the region. As a result, amenu 1708 may be displayed and may include several alternate recognitionresults based, at least partially, on recognizing atoms 1704 inisolation. The user may select alternate recognition result 1710 as acorrection hint. Atoms 1704 include a stroke inside an expression undera root symbol. Therefore, rule 1 is satisfied. Because rule 1 issatisfied, all strokes of atoms 1704 are to be included in theexpression under the root symbol. Thus, the correction hint indicatesthat all of atoms 1704 are to be included under the root symbol andatoms 1704 correspond to “z²”. Input atoms 1702 may be re-recognized,respecting the correction hint, to produce correct recognition result1802 (FIG. 18).

Exemplary Processes

FIG. 19 illustrates a flowchart of an exemplary process for correcting amisrecognition, which may be performed in embodiments consistent withthe subject matter of this disclosure. The process may begin with aprocessing device, such as, for example, processing device 100,receiving input handwritten atoms, or strokes (act 1902). The processingdevice may then recognize the input atoms or strokes (act 1904).

FIGS. 20-21 are flowcharts illustrating an exemplary process forperforming act 1904 in embodiments consistent with the subject matter ofthis disclosure. The process may begin with the processing devicedetermining whether any correction hints indicate an ink parsing error(act 2002). A correction hint may indicate an ink parsing error when asymbol associated with the correction hint was not originally consideredas part of a possible recognition result, with respect to one or moreatoms of the correction hint. If the processing device determines that acorrection hint indicates an ink parsing error, then the processingdevice may apply the correction and may shrink the one or more atomsassociated with the correction hint (act 2004). This may be applied withrespect to any correction hint indicating an ink parsing error.

FIG. 22 is a flowchart illustrating exemplary processing with respect toact 2004 in an embodiment consistent with the subject matter thisdisclosure. The process may begin with l, r, t, and b being set to aleft coordinate, a right coordinate, a top coordinate, and a bottomcoordinate, respectively, of a first bounding box defining a region R(act 2202). The first bounding box surrounds one or more atomscorresponding to an ink parsing error. A bounding box may be defined asa minimal rectangle which contains a whole region or a whole documentatom. In some embodiments, the coordinates may be relative to aCartesian coordinate system.

Next, l1, r1, t1, and b1 may be set to l, r, t, b, respectively (act2204). Coordinates l1, r1, t1, and b1 may be a left coordinate, a rightcoordinate, a top coordinate, and a bottom coordinate, respectively, ofa shrunken region, which may originally be set to coordinates of thefirst bounding box.

Next, an atom A in document D, but not in region R may be taken, orselected (act 2205). Coordinates l2, r2, t2, and b2 may be set to a leftcoordinate, a right coordinate, a top coordinate, and a bottomcoordinate, respectively, of an atom bounding box of the atom A (act2206).

L may then be set to a maximum value of l1 and l2, R may be set to aminimum value of r1 and r2, T may be set to a minimum value of t1 andt2, and B may be set to a maximum value of b1 and b2 (act 2208). A checkmay then be performed to determine whether L≦R and T≦B (act 2210). IfL>R or T>B, then shrinking of the region may not be performed.Otherwise, the bounding boxes may overlap one another and minimalshrinking of the first bounding box may be performed. Thus, if L≦R andT≦B, then a check may be performed to determine if (r1−L)<MIN(R−l1,T−t1, b1−B) (act 2212). If (r1−L)<MIN(R−l1, T−t1, b1−B), then r1 may beset to L (act 2216).

If, during act 2212, the check determines that (r1−L)≧MIN(R−l1, T−t1,b1−B), then a check may be performed to determine if (R−l1)<MIN(T−t1,b1−B) (act 2218). If (R−l1)<MIN(T−t1, b1−B), then l1 may be set to R(act 2222).

If, during act 2218, the check determines that (R−l1)≧MIN(T−t1, b1−B),then a check may be performed to determine if (T−t1)<(b1−B) (act 2224).If (T−t1)<(b1−B), then t1 may be set to B (act 2228).

If, during act 2224, (T−t1) is determined to be ≧(b1−B), then b1 may beset to T (act 2232).

A next atom A, not in region R may then be taken, or selected (act2234). A check may be performed to determine if the next atom A exists,or is found (act 2236). If the next atom A is found then acts 2206-2236may be repeated. Otherwise the process is completed.

As a result of performing the process of FIG. 22, the shrunken boundingbox may be strictly inside an area defined by the first bounding box asoriginally defined.

Returning to FIG. 20, after determining that no correction hintsindicate a parsing error, or after determining that one or morecorrection hints indicate a parsing error and shrinking of one or moreatoms has been performed, then the processing device may apply arewriting rule of a grammar to one or more atoms (act 2006). Theprocessing device may then determine whether the applied rewriting ruleconforms to, or respects, all of the correction hints (act 2008).

FIGS. 23 and 24 are flowcharts illustrating exemplary processing fordetermining whether a rewriting rule respects, or conforms to, acorrection hint in embodiments consistent with the subject matter ofthis disclosure. The process may begin with the processing devicedetermining whether the rewriting rule is unary (i.e. R_(⊥1)≠0,R_(⊥2)=0)or whether the correction hint is unary (i.e., CR_(⊥1)≠0, CR_(⊥2)=0)(act 2302). If the rewriting rule is unary or the correction hint isunary, then an indication may be set indicating that the rewriting ruleconforms to, or respects, the correction hint (act 2304) and the processmay be complete.

If, during act 2302, the processing device determines that the rewritingrule is not unary and the correction hint is not unary, then theprocessing device may determine whether the rewriting rule is terminal(i.e. R_(⊥1)=0,R_(⊥2)=0) (act 2306).

If, during act 2306, the rewriting rule is determined to be terminal,then processing device may determine whether the rewriting rule and thecorrection hint are disjoint (i.e. R₀∩CR₀=0) (act 2308). If therewriting rule and the correction hint are disjoint, then an indicationmay be set indicating that the rewriting rule conforms to, or respects,the correction hint (act 2304) and the process may be complete.

If, during act 2308, the processing device determines that the rewritingrule and the correction hint are not disjoint, then the processingdevice may determine if the correction hint is binary (i.e.CR_(⊥1)≠0,CR_(⊥2)≠0) (act 2310). If the correction hint is determined tobe binary, then the processing device may determine if a region of theterminal rewriting rule is a subset of one of two regions from aright-hand side of a definition of the correction hint (i.e. R₀ ⊂CR₁

R₀ ⊂CR₂) (act 2312). If so, then an indication may be set indicatingthat the rewriting rule conforms to, or respects, the correction hint(act 2304) and the process may be complete.

If, during act 2312, the region of the terminal rewriting rule is not asubset of one of two regions from the right-hand side of the definitionof the correction hint, then an indication may be set indicating thatthe rewriting rule does not conform to, or does not respect, thecorrection hint (act 2316) and the process may be complete.

If, during act 2310, the processing device determines that thecorrection hint is not binary, then the processing device may determineif the correction hint is terminal (i.e. CR_(⊥1)=0,CR_(⊥2)=0) and therewriting rule and the correction hint are equal (act 2314). If so, thenan indication may be set indicating that the rewriting rule conforms to,or respects, the correction hint (act 2304) and the process may becomplete. Otherwise, if, during act 2314, the correction hint is notterminal or the rewriting rule is not equal to the correction hint, thenan indication may be set indicating that the rewriting rule does notconform to, or does not respect, the correction hint (act 2316) and theprocess may be complete.

If, during act 2306, the processing device determines that the rewritingrule is not terminal, then the processing device may determine whetherthe rewriting rule is binary (i.e. R_(⊥1)≠0,R_(⊥2)≠0) (act 2402; FIG.24). If, during act 2402, the processing device determines that therewriting rule is not binary, then an indication may be set indicatingthat the rewriting rule does not conform to, or does not respect, thecorrection hint (act 2414) and the process may be complete.

If, during act 2402, the processing device determines that rewritingrule is binary, then the processing device may determine whether therewriting rule and the correction hint are disjoint (i.e. R₀∩CR₀=0) (act2406). If the rewriting rule and the correction hint are determined tobe disjoint, then an indication may be set indicating that the rewritingrule conforms to, or respects, the correction hint (act 2408) and theprocess may be complete.

If, during act 2406, the processing device determines that the rewritingrule and the correction hint are not disjoint, then the processingdevice may determine if the correction hint is binary (i.e.CR_(⊥1)≠0,CR_(⊥2)≠0) and a main region of the binary rewriting rule is asubset of one of two regions of a right-hand side of the correction hintor a main region of the correction hint is a subset of one of tworegions on a right-hand side of the rewriting rules (i.e. R₀ ⊂CR₁

R₀ ⊂CR₂

CR₀ ⊂R₁

CR₀ ⊂R₂) (act 2410). If so, then an indication may be set indicatingthat the rewriting rule conforms to, or respects, the correction hint(act 2408) and the process may be complete.

If, during act 2410, the processing device determines that the conditionis not true, then the processing device may determine whether thecorrection hint is terminal (i.e. CR_(⊥1)=0,CR_(⊥2)=0) and a main regionof the correction hint is a subset of one of two regions on a right-handside of the binary rewriting rule (i.e. CR₀ ⊂R₁

CR₀ ⊂R₂) (act 2412). If the condition checked during act 2412 is true,then an indication may be set indicating that the rewriting ruleconforms to, or respects, the correction hint (act 2408) and the processmay be complete. Otherwise, an indication may be set indicating that therewriting rule does not conform to, or does not respect, the correctionhint (act 2414) and the process may be complete.

Returning to FIG. 20, if the rewriting rule does not conform to, or doesnot respect correction hint(s), then a penalty score may be added to asum of rewriting rules applied to produce a possible recognition result(act 2010). If the rewriting rule conforms to, or respects, thecorrection hint(s), then a rewriting rule score (as may be assigned by arecognizer) may be added to the sum of rewriting rules applied toproduce the possible recognition result (act 2012).

After performing act 2012 or act 2010, the processing device maydetermine whether there are additional productions which may be producedby applying more rewriting rules (act 2014). If additional productionsmay be produced, then acts 2006-2014 may again be performed. If noadditional productions may be produced (i.e. all atoms are assigned toterminal productions), then the processing device may save the sum as ascore of the possible recognition result and may save the possiblerecognition result (act 2016).

The processing device may then determine whether additional sequences ofrewriting rules may be applied to the input atoms (act 2018). Ifadditional productions sets may be produced, then the processing devicemay choose a rewriting rule to apply to the input atoms to start a newsequence of rewriting rules (act 2020). Acts 2006-2020 may again beperformed.

If, during act 2018, the processing device determines that no additionalsequences of rewriting rules may be produced, then the processing devicemay select, as a recognition result, one of a number of possiblerecognition results corresponding to a best score (act 2102; FIG. 21).As mentioned previously, a best score may be a highest score. However,in some embodiments, a best score may be a lowest score.

Returning to FIG. 19, the processing device may then provide, ordisplay, the selected recognition result (act 1906). The processingdevice may then determine whether the process is done (act 1908). Theprocess may be indicated as done when a user enters a particular input,after a predetermined amount of time has passed with no input beingentered, or by any number of other actions or conditions.

If the process is not done, then the processing device may receive asecond input indicating a misrecognition of at least one atom (act1910). The second input may be a selection of one or more atomscorresponding to the misrecognition, such that the selection defines aregion including the one or more atoms. The processing device may thenattempt to recognize, in isolation, the selected one or more atoms inthe defined region (act 1912). The processing device may then provide,or display, a number of possible recognition results (act 1914). Theprocessing device may receive a selection of one of the possiblerecognition results as a correction hint (act 1916). Acts 1904-1916 mayagain be repeated.

Conclusion

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter in the appended claims is not necessarilylimited to the specific features or acts described above. Rather, thespecific features and acts described above are disclosed as exampleforms for implementing the claims.

Although the above descriptions may contain specific details, they arenot to be construed as limiting the claims in any way. Otherconfigurations of the described embodiments are part of the scope ofthis disclosure. Further, implementations consistent with the subjectmatter of this disclosure may have more or fewer acts than as describedin FIGS. 20-24, or may implement acts in a different order than as shownin FIGS. 20-24. Accordingly, the appended claims and their legalequivalents define the scope of the invention, rather than any specificexamples given.

1. A machine-implemented method for correcting a recognized handwrittenmathematical expression, the machine-implemented method comprising:receiving input indicating a region having at least one atom of aplurality of atoms corresponding to a misrecognized production of arecognized mathematical expression; providing a menu of possiblerecognition results based on the at least one atom in the region;receiving a selection of one of the possible recognition results as acorrection hint; shrinking the region and linearly transforming the atleast one atom corresponding to the correction hint such that the atleast one atom is horizontally or vertically separate from other atomsof the plurality of atoms; and re-recognizing the plurality of atoms,including the at least one linearly transformed atom, to produce acorrectly recognized mathematical expression.
 2. The machine-implementedmethod of claim 1, wherein: the re-recognizing of the plurality ofatoms, including the at least one linearly transformed atom, furthercomprises: adding a penalty score to each applied one of a plurality ofrewriting rules which does not respect the correction hint, andselecting a best recognition result from among a plurality of possiblerecognition results based on a sum of scores assigned to each one of theplurality of rewriting rules applied to produce each respective one ofthe plurality of possible recognition results.
 3. Themachine-implemented method of claim 1, wherein: the region is indicatedby a bounding box, and the shrinking of the region produces a minimalbounding box.
 4. The machine-implemented method of claim 1, wherein: theregion is indicated by a bounding box, and the shrinking of the regionfurther comprises: determining whether the bounding box bounding the atleast one atom corresponding to the correction hint, overlaps a secondbounding box bounding at least one of the others of the plurality ofatoms, and shrinking the bounding box to produce a shrunken bounding boxwhen the bounding box and the second bounding box are determined tooverlap, the shrunken bounding box being a minimal bounding boxseparated from the second bounding box.
 5. The machine-implementedmethod of claim 1, further comprising: receiving a second inputindicating a second region including at least one atom of the pluralityof atoms corresponding to a misrecognized symbol included in a matrix;providing a second menu of second possible recognition results based onthe at least one atom in the second region; receiving a second selectionof one of the second possible recognition results as a second correctionhint; determining whether the matrix has an over-grouping error based onthe second correction hint; and leveraging the second correction hint tocorrect the over-grouping error.
 6. The machine-implemented method ofclaim 1, further comprising: receiving a second input indicating asecond region having a second plurality of atoms of the plurality ofatoms corresponding to a portion of the handwritten mathematicalexpression, at least some of the second plurality of atoms having beenmisrecognized as a symbol not being under a root symbol; providing asecond menu of second possible recognition results based on the secondplurality of atoms; receiving a second selection of one of the secondpossible recognition results as a second correction hint; determiningwhether a first part of the second plurality of atoms corresponding tothe second correction hint is under a root structure and whether asecond part of the second plurality of atoms corresponding to the secondcorrection hint is not under the root structure; and recognizing thesecond plurality of atoms, according to the second correction hint, asat least one symbol under the root structure when the first part of thesecond plurality of atoms is determined to be under a root structure andthe second part of the second plurality of atoms is determined to be notunder the root structure.
 7. The machine-implemented method of claim 1,further comprising: receiving a second input indicating a second regionhaving at least one atom of the plurality of atoms corresponding to amisrecognized portion of the handwritten mathematical expression;providing a second menu of second possible recognition results based onthe at least one atom of the plurality of atoms corresponding to themisrecognized portion of the handwritten mathematical expression, thesecond possible recognition results being among a top plurality ofscores calculated by summing scores assigned to each one of a pluralityof rewriting rules applied to produce each respective one of theplurality of second possible recognition results; receiving a secondselection of one of the second possible recognition results as a secondcorrection hint; adding a penalty score to each one of the plurality ofrewriting rules which does not respect the second correction hint; andre-recognizing the plurality of atoms of the handwritten mathematicalexpression, the re-recognizing further comprising: selecting a bestrecognition result from among a third plurality of possible recognitionresults based on a sum of scores assigned to each one of the pluralityof rewriting rules applied to produce each respective one of the thirdplurality of possible recognition results.
 8. A processing device forrecognizing a plurality of atoms of structured handwritten input, theprocessing device comprising: at least one processor; and a memoryconnected to the at least one processor, the memory further comprising:instructions for receiving an input indicating a second plurality ofatoms of the plurality of atoms corresponding to a portion of thestructured handwritten input, at least some of the second plurality ofatoms having been misrecognized as a symbol not being under a rootstructure, instructions for displaying a menu of possible recognitionresults based on the second plurality of atoms, instructions forreceiving a selection of one of the possible recognition results as acorrection hint, instructions for determining whether a first part ofthe second plurality of atoms, corresponding to the correction hint, isunder the root structure and whether a second part of the secondplurality of atoms, corresponding to the correction hint, is not underthe root structure; and instructions for recognizing the plurality ofatoms, in conformance with the second correction hint, as including atleast one symbol under the root structure, corresponding to the firstpart of the second plurality of atoms and the second part of the secondplurality of atoms, when the first part of the second plurality of atomsis determined to be under the root structure and the second part of thesecond plurality of atoms is determined to be not under the rootstructure.
 9. The processing device of claim 8, wherein: theinstructions for recognizing the plurality of atoms, in conformance withthe second correction hint, further comprise: instructions for producinga second plurality of possible recognition results by summing scorescorresponding to ones of a plurality of rewriting rules applied whenproducing each of the ones of the second plurality of possiblerecognition results from the plurality of atoms, instructions fordetermining whether applying any of the ones of the plurality ofrewriting rules to the plurality of atoms to produce the ones of thesecond plurality of possible recognition results conforms with thecorrection hint, instructions for adding a penalty score to ones of thesums of the scores having a corresponding rewriting rule not conformingto the correction hint, and instruction for selecting a best recognitionresult from the second plurality of possible recognition results, thebest recognition result having a highest score among the summed scoresof each of the ones of the second possible recognition results.
 10. Theprocessing device of claim 9, wherein: the instructions for determiningwhether applying any of the ones of the plurality of rewriting rules tothe plurality of atoms to produce the ones of the second plurality ofpossible recognition results conforms with the correction hint furthercomprises: instructions for determining that applying a rewriting ruleconforms with the correction hint when the rewriting rule is unary orthe correction hint is unary, instructions for determining that applyingthe rewriting rule conforms with the correction hint when the rewritingrule is terminal and (i) the rewriting rule is disjoint with thecorrection hint, (ii) the correction hint is binary and a region of therewriting rule is a subset of one of two regions of the correction hint,or (iii) when the rewriting rule is equivalent to the correction hint,and instructions for determining that applying the rewriting ruleconforms with the correction hint when the rewriting rule is binary and(i) the rewriting rule is disjoint with the correction hint, (ii) thecorrection hint is binary and either a main region of the rewriting ruleis a subset of one of the two regions of the correction hint or a mainregion of the correction hint is a subset of one of the two regions ofthe rewriting rule, or (iii) the correction hint is terminal and themain region of the correction hint is a subset of one of the two regionsof the rewriting rule.
 11. The processing device of claim 8, wherein thememory further comprises: instructions for receiving a second inputindicating at least one atom of the plurality of atoms; instructions fordisplaying a second menu including alternate recognitions of the atleast one atom, each of the alternate recognitions having a respectivescore being lower than an original score corresponding to an originalrecognition with respect to the at least one atom of the plurality ofatoms; instructions for receiving a second selection of one of thealternate recognitions as a second correction hint; instructions forproducing a second plurality of possible recognition results based onsummed scores corresponding to ones of a plurality of rewriting rulesapplied when producing each of the ones of the second plurality ofpossible recognition results from the plurality of atoms; instructionsfor determining whether applying any of the ones of the plurality ofrewriting rules to the plurality of atoms to produce the ones of thesecond plurality of possible recognition results conforms with thesecond correction hint; instructions for adding a penalty score to onesof the sums of the scores having a corresponding rewriting rule notconforming to the second correction hint; and instruction for selectinga best recognition result from the second plurality of possiblerecognition results, the best recognition result having a highest scoreamong the summed scores of each of the ones of the second possiblerecognition results.
 12. The processing device of claim 8, wherein thememory further comprises: instructions for receiving a second inputindicating a second plurality of atoms of the plurality of atomscorresponding to at least one misrecognized symbol included in a matrix;instructions for displaying a second menu of second possible recognitionresults based on the indicated second plurality of atoms; instructionsfor receiving a second selection of one of the second possiblerecognition results as a second correction hint; instructions fordetermining whether the second correction hint indicates that the secondplurality of atoms is to be in a single matrix cell; and instructionsfor leveraging the second correction hint to correct an over-groupingerror when the second correction hint is determined to indicate that thesecond plurality of atoms is to be in a single matrix cell.
 13. Theprocessing device of claim 12, wherein the instructions for leveragingthe second correction hint to correct an over-grouping error when thesecond correction hint is determined to indicate that the secondplurality of atoms is to be in a single matrix cell further comprise:instructions for producing a third plurality of possible recognitionresults by summing scores corresponding to ones of a plurality ofrewriting rules used when producing each of the ones of the thirdplurality of possible recognition results from the plurality of atoms;instructions for determining whether applying any of the ones of theplurality of rewriting rules to the plurality of atoms to produce theones of the third plurality of possible recognition results conformswith the second correction hint; instructions for adding a penalty scoreto ones of the sums of the scores having a corresponding rewriting rulenot conforming to the second correction hint; and instruction forselecting a best recognition result from the third plurality of possiblerecognition results.
 14. A tangible machine-readable medium havinginstructions for at least one processor recorded thereon, theinstructions comprising: instructions for receiving input indicating atleast one atom of a plurality of atoms of handwritten input, the atleast one atom having been misrecognized by a recognizer; instructionsfor displaying a menu of possible recognition results based on the atleast one atom; instructions for receiving a selection of one of thepossible recognition results as a correction hint; and instructions forleveraging the correction hint to produce a corrected recognitionresult, the instructions for leveraging the correction hint to produce acorrected recognition result further comprising at least one group ofinstructions selected from: a first group of instructions for linearlytransforming the at least one atom corresponding to the correction hintsuch that the at least one atom is horizontally or vertically separatefrom other atoms of the plurality of atoms, a second group ofinstructions for leveraging the correction hint to correct anover-grouping error of a matrix when the second correction hintindicates that the at least one atom is to be in a single cell of amatrix, and a third group of instructions for recognizing the at leastone atom as being under a root structure when the correction hintindicates that a first part of the at least one atom corresponding tothe correction hint is under a root structure and a second part of theat least one atom corresponding to the correction hint is not under theroot structure.
 15. The tangible machine-readable medium of claim 14,wherein the instructions for leveraging the correction hint to produce acorrected recognition result further comprise at least two groups ofinstructions selected from: the first group of instructions, the secondgroup of instructions, and the third group of instructions.
 16. Thetangible machine-readable medium of claim 14, wherein the instructionsfor leveraging the correction hint to produce a corrected recognitionresult further comprise: instructions for producing a plurality ofsecond possible recognition results by summing scores corresponding toones of a plurality of rewriting rules used when producing each one ofthe plurality of second possible recognition results from the pluralityof atoms; instructions for determining whether applying any of the onesof the plurality of rewriting rules to the plurality of atoms to producethe ones of the plurality of second possible recognition results doesnot conform with the correction hint; instructions for adding a penaltyscore to ones of the sums of the scores having a corresponding rewritingrule not conforming with the correction hint; and instruction forselecting a best recognition result from the plurality of secondpossible recognition results, based on the summed scores of each of theones of the plurality of second possible recognition results.
 17. Thetangible machine-readable medium of claim 16, wherein the instructionsfor determining whether applying any of the ones of the plurality ofrewriting rules to the plurality of atoms to produce the ones of theplurality of second possible recognition results does not conform withthe correction hint further comprise: instructions for determining thatapplying a rewriting rule conforms with the correction hint when therewriting rule is unary or the correction hint is unary, instructionsfor determining that applying the rewriting rule conforms with thecorrection hint when the rewriting rule is terminal and (i) therewriting rule is disjoint with the correction hint, (ii) the correctionhint is binary and a region of the rewriting rule is a subset of one oftwo regions of the correction hint, or (iii) the rewriting rule isequivalent to the correction hint, and instructions for determining thatapplying the rewriting rule conforms with the correction hint when therewriting rule is binary and (i) the rewriting rule is disjoint with thecorrection hint, (ii) the correction hint is binary and either a mainregion of the rewriting rule is a subset of one of the two regions ofthe correction hint or a main region of the correction hint is a subsetof one of the two regions of the rewriting rule, or (iii) the correctionhint is terminal and the main region of the correction hint is a subsetof one of the two regions of the rewriting rule.
 18. The tangiblemachine-readable medium of claim 14, wherein: the instructions forleveraging the correction hint to produce a corrected recognition resultfurther comprise the first group of instructions, the received inputdefines a bounding box surrounding the at least one atom, and the firstgroup of instructions further comprise: instructions for determiningwhether the bounding box bounding the at least one atom corresponding tothe correction hint overlaps any other atoms not part of the correctionhint, and instructions for shrinking the bounding box to produce ashrunken bounding box when the bounding box is determined to overlap anyof the other atoms not part of the correction hint, the shrunkenbounding box not overlapping any of the other atoms, and the at leastone atom being linearly transformed in proportion to the shrinking ofthe bounding box.
 19. The tangible machine-readable medium of claim 18,wherein the instructions for shrinking the bounding box to produce ashrunken bounding box when the bounding box is determined to overlap anyof the other atoms not part of the correction hint further comprise:instructions for shrinking the bounding box no more than a minimalamount such that the bounding box does not overlap with any of the otheratoms.
 20. The tangible machine-readable medium of claim 18, wherein theshrunken bounding box is strictly inside an area defined by the boundingbox.